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I.  PROGRESS 

Progress  during  the  mid-March  to  mid- June  time  frame  was  minimal  since 
the  anticipated  incremental  funding  was  not  received  until  mid- June  and  since 
the  program  was  redirected  from  optical  systolic  array  processor  outer  product 
applications  to  associative  memory  applications . 

Dr.  John  Gruninger  and  Mr.  Scott  Israel  of  Aerodyne  visited  NRL  on  April 
2  in  order  to  participate  in  the  redirection  of  the  Aerodyne  effort.  This 
redirection  process  was  completed  on  June  21  with  a  joint  meeting  of  NRL  and 
Aerodyne  personnel  at  Aerodyne. 

It  was  agreed  that  Aerodyne  efforts  for  the  rest  of  FY’85  would  be 
focused  on  the  investigation  of  iterative  algorithms  for  possible  optical 
implementation  of  nonlinear  novelty  filters.  A  first  cut  discussion  of 
relevant  concepts  has  been  forwarded  to  Dr.  Lee  and  is  attached  as  an  addendum 
to  this  report.  (See  Addendum  No.  1) 

A  report  on  approximate  singular  value  decomposition  techniques  for  ill- 
conditioned  matrices  was  prepared  and  presented  at  the  OSA  Special  Meeting  on 


Optical  Computing  in  Reno,  NV.  in  March.  A  copy  of  the  vugraph  presentation  is 
attached.  (See  Addendum  No.  2) 

! 

;•  II.  FUTURE  WORK 

Drs.  Gruninger  and  Putnam  will  continue  algorithm  development  for  novelty 

» 

1  filter  applications.  They  will  visit  NRL  during  the  next  reporting  period  to 

U  present  progress  to  that  point. 

to 

III.  FINANCIAL  STATUS 

Spending  for  this  period  (four  months)  -  $14,166 
■  Cumulative  Spending  (through  July  5,  1985)  -  $59,309. 
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ADDENDUM  NO.  1 


AERODYNE  RESEARCH.  Irtc 

45  Manning  Road 

The  Research  Center  at  Manning  Park 
Billerica,  Massachusetts  01821  (617)663-9500 


July  15, 


Dr.  John  Lee 

Naval  Research  Laboratory 
Washington,  D.C.  20375. 


Dear  John, 

Here  are  ray  thoughts  and  materials  for  further  efforts. 

I  have  Included  work  in  rough  draft  form  on  the  incorporation  of  weights 
and  constraints  into  projection  operators.  As  I  mentioned  to  you  earlier,  I 
have  been  working  on  subspace  methods  and  oblique  projection  methods  for  some 
time  now.  The  draft  on  weights  shows  that  weighting  can  be  used  to  improve 
the  condition  number  of  a  memory  matrix.  A  general  method  for  weight  selection 
may  be  of  considerable  use  in  making  recall  less  sensitive  to  noise.  The 
connection  between  constraints,  generalized  inverses  of  several  varieties  and 
subspace  methods  is  briefly  outlined  in  the  material  labeled  Generalized  Inverses 
and  Constrained  Projections.  There  are  some  algorithms  developed  that  are  similar 
to  the  Greville  algorithm  for  the  Moore  Penrose  Inverse.  These  should  be 
implementable  in  optical  hardware.  These  algorithms  have  been  used  for  image 
reconstruction  and  restoration. 


The  viewgraph  material  shows  an  interesting  example  of  constrained  and 
weighted  projection  techniques  in  use  as  a  novelty  filter.  This  work  was  done 
some  time  ago  for  a  different  purpose  and  was  never  pursued.  The  example  filter 
expects  a  spectrum  which  is  a  member  of  one  of  eight  classes  of  compounds  or  is  a 
mixture  of  members  of  the  eight  classes.  The  classes  are: 


aliphatic  hydrocarbons  R 

aliphatic  alcohols  ROH 

aliphatic  chlorides  _  RC1 

aliphatic  chloro  alcohols  RC10H 

aromatic  hydrocarbons  to 

aromatic  alcohols  $0H 

aromatic  chlorides  0C1 

aromatic  chloro  alcohols  AC10H 


The  example  unknown  that  was  testing  the  filter  was  a  spectrum  of  2-Chloro, 

U  Nitro  Benzyl  Alcohol,  an  aromatic  nitro  chloro  alcohol.  The  unknown  is  similar 
to  the  classes  present  but  differs  in  having  a  nitro  group.  Hence  there  should  be 
a  novelty. 


Dr.  John  Lee 
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Three  calculations  were  performed  and  reported  in  the  Table  on  the  third  view- 
graph  of  the  package.  The  first  of  the  three  calculations  are  for  a  standard 
Novelty  filter  using  a  Moore  Penrose  pseudo  inverse.  The  second  calculation 
included  constraints  to  keep  the  components  of  the  estimated  spectra  positive. 

The  third  calculation  included  these  constraints  as  well  as  a  weighting  based 
on  the  unknown  itself.  This  last  Novelty  filter  is  not  only  nonlinear  but  also 
differs  from  unknown  to  unknown. 

The  first  filter  finds  the  Novelty  as 

(l- 20^)1/ 

where  N  is  the  Novelty,  U  is  the  unknown  and  0^  is  an  Oblique  projection  for  the 

kth  class.  P  »  SO  is  an  orthogonal  projection.  This  filter  is  the  standard 
k  k 

linear  novelty  filter. 

The  second  filter  has  constraints 

(/-2o„)u 

U  -  Sac 

The  third  filter  has  constraints  and  is  weighted  with  the  unknown 

fr-ZOk{u))Ct 

K 

is  a  function  of  the  current  unknown. 

The  cable  of  results  gives  the  lengths  of  the  component  vectors  of  the  sub¬ 
spaces.  The  "Noise"  is  the  Novelty.  The  negative  numbers  appearing  for  the 
unconstrained  filter  are  a  result  of  the  filter  both  adding  and  subtracting 
components  to  attempt  to  match  the  novelty. 

Both  the  constrained  and  the  weighted  and  constrained  filter  do  a  much  better 
job.  Almost  the  entire  unknown  spectrum  is  recovered  by  the  weighted  novelty  filter 
For  the  unknown  normalized  to  unit  length  the  length  of  the- noise  (novelty)  is  .93. 

Efforts  over  the  next  few  months  will  focus  on  Subspace  method  of  Associative 
Recall.  Key  areas  of  interest  include: 

•  Nonlinear  novelty  filters. 

Introduction  of  nonlinearity  through  constraints  both  on  known  and 
unknown.  Introduction  of  nonlinearity  through  input  pattern  weighting 
of  filter. 
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•  Investigate  iterative  algorithms  for  possible  optical 
implementation  of  nonlinear  novelty  filters. 

Efforts  in  the  Fall  will  include: 

•  Recall  from  partial  pattern  matching. 

Develop  analysis  for  effects  of  noise  and  near  linear  dependencies 
on  partial  pattern  matching.  Investigate  effects  of  weighting  on 
reducing  apparent  linear  dependencies . 

I  think  this  is  an  important  area  where  analysis  is  needed.  Recent  work  has 
been  based  on  Monte  Carlo  models.  (See  G.  S.  Stiles  and*  Dong  Lih  Denq  "On  the 
Effect  of  Noise  on  the  Moore  Penrose  Generalized  Inverse  Associative  Memory", 
IEEE  TRANS  PAM  I,  2.  358  (1985)). 

I  hope  this  material  will  be  of  help  when  Bob  Carter  comes  to  see  you.  I 
will  be  in  Eglin,  Florida  Thursday  and  Friday.  I  will  give  you  a  call  and  let 
you  know  how  to  reach  me  If  you  have  questions  then.  I  will  have  difficulty 
arranging  a  trip  to  NRL  until  after  the  1st  of  August.  I  hope  early  in  August 
will  be  alright. 


Sincerely, 


Senior  Applied  Mathematician. 


Yair  Censor 


References  on  Iterative  Algorithms 


Finite  Series  Expansion  Reconstruction  Method,  Proc.  IEEE,  7_1>  409  (1983) 
Arnold  Lent  and  Yair  Censor 

"Extensions  of  Hildreth's  Row  Action  Method  for  Quadratic  Programming", 
SIAM  J.  Control  and  Optimization,  1_8,  444  (1980). 

Ronald  Schafer,  Russell  Mersereau  and  Mark  Richards 

Constrained  Iterative  Restoration  Algorithms,  Proc.  IEEE  69,  432  (1981). 

Tommy  Elfving 

On  Some  Methods  of  Entropy  Maximization  and  Matrix  Scaling 
Lin.  Algebra  and  its  Applications,  34,  321  (1980). 


CONSTRAINED  PROJECTION  OPERATORS 


The  representation  of  a  class  as  a  subspace  of  its  features  does  not 
always  contain  all  of  the  information  that  in  known  about  the  class.  For 
example  with  spectral  classes,  the  absorbances  are  always  positive,  but  the 
•ubspaces  spanned  by  a  set  of  spectra  include  both  positive  and  negative 
absorbances.  In  order  to  avoid  nonphysical  spectral  estimates  as  well  as 
Incorrect  classifications,  it  is  useful  to  constrain  the  absorbances  to  be 
positive.  We  develop  below  the  projection  method  for  two  types  of 
constraints,  inequality  constraints  and  equality  constraints.  The  constraint 
methods  are  developed  in  the  framework  of  constrained  least  squares.  The 
result  is  a  set  of  operators  for  constrained  oblique  projection* 

The  solution  to  the  least  square  problem  can  be  expressed  as 
minimize  (b-Ax)T(b-Ax) 

A 

with  respect  to  z.  The  solution  Ax  Is  given  by 

Ax  -  -  Pb 

where  P  Is  the  orthogonal  projector.  The  inclusion  of  constraints  can  be 
accomplished  using  Lagrangian  techniques.  Constraints  of  the  form  Gx  -  0, 
equality  constraints  and  Gx  £  0  inequality  constraints  will  be  considered. 

The  more  general  constraints  relations  Gx  _>  h  can  be  converted  to  the  above  by 
the  transformation  x  -  y  +  G*h  where  G1  is  the  pseudoinverse  of  G. 

Equality  Constraints 

The  least  square  method  with  equality  constraints  can  be  expressed  as 


1 


T 

minimize  (b-Ax)  (b-Ax) 

subject  to  Gx  •  0. 

The  associated  Lagrangian  is 

ITT  T  T 

L(x,A)  -  j  x  A  Ax  -  x  A  b  -  AGx 

A  saddlepolnt  solution  of  L(x,A)  is  obtained  if 

T  T  T 

A  Ax  +  A  b  -  G  A  -  0 

Gx  -  0 

and  A  £  0 

A  Is  the  vector  of  Lagrange  Multipliers. 

The  solution  x  of  the  constrained  problem  can  be  given  In  terms  of  the 
unconstrained  problem  as 

*  T  —IT 

x  -  X  +  (A1  A)  Va 

where  A  is  given  by 

A  -  -  (G(ATA)“lGT)  Gx 
The  solution  Ax  can  be  expressed  as 

Ax  -  A  QCATA)"1Ab  -  PQb 


2 


where  Q  -  1  -  <ATA)“1GT(G(ATA)"1C  ]  C 

A  bit  of  algebra  will  reveal  that  Q  is  idempotent  and  hence  a  projector. 

2  2 
Q  •  Q  and  further  Pq  ■ 

m 

Q  projects  the  unrestricted  solution  x  onto  the  constraint  subspace. 

The  residual  includes  the  unrestricted  residual  as  well  as  the  residual 
arising  from  projection  onto  the 


AQ  -  A(l-Q)(ATA)"lATb 

subspace  that  violates  the  constraints. 

Inequality  Constraints 

For  inequality  constraints  the  optimization  problem  is 

Minimize  (b-Ax)^(b-Ax) 

Subject  to  Gx  >_  0 

Following  the  method  outlined  for  equality  constraints  the  solution  for  x  and 
X  are  obtained  from  the  solution  of 

T  T  T 

(A  A)X  -  A  b  -  G  A  -  0 

J 


Gx  >  0 


XGx  -  0 

and  A  >_  0 

The  major  difference  between  equality  and  inequality  constraints  is  that 

A 

the  inequality  constraints  need  not  be  active.  If  x  satisfies  the  constraints 
in  that 


Gx  >  0  then  x  »  x 

A  A 

A  *  0  is  the  solution.  In  general  for  every  component  x£  of  x  that  is 
greater  than  zero,  the  corresponding  Lagrange  multiplier  X±  is  zero.  The 
nonzero  A*.  corresponds  to  active  constraints  ■  0.  These  correspond  to 

A  A 

the  unrestricted  solutions  xi  violating  the  constraints  that  is  G^x^ 

<  0.  The  nonzero  A*  are  given  as  before  x,  X  end  Pq  have  the  same 
properties.  Q  however  depends  on  b,  if  for  example 

Gx  -  G(ATA)’1ATb  >  0  then  Q  -  1  . 

Without  loss  of  generality  ve  can  assume  that  the  first  k  constraints  are 
inactive  and  the  remaining  1-k  constraints  are  active.  Then  G  can  be 
partitioned  into  [G^Gj]^  where  Gj  corresponds  to  the  active  constraints  and 
the  nonzero  Lagranglan  components  solved  for  from 


x  -  -  (g2<ata)"1g£)  g2  x 

The  projection  operates  Pq  for  the  constrained  least  square  problem  can  be 
considered  as  a  sum  of  oblique  projection  operators,  one  for  each  class  as  in 
the  case  of  unconstrained  least  squares. 


4 


relation  between  generalized  inverses  and  projection  methods 


A  relationship  between  certain  generalized  inverses  and  constrained 
oblique  projection  operators  was  Identified.  The  varied  of  generalized 
inverses  suggests  that  a  large  variety  of  potentially  useful  projection 
operators  can  be  generated. 

Oblique  projectors  and  constraints  can  be  considered  in  terns  of 
generalized  inverses  of  as  trices.  A  projection  operator  can  be  written  as 
Pw  •  w  w*  where  w*  is  the  generalized  inverse  of  w.  The  selection  of 
the  Moore  Penroae  inverse 


1  ,  T  %I  T  ,  T  x-1  T 

w  ■  (w  w)  w  •  (w  w)  w 


-1  T 
A  w 


leads  to  the  orthogonal  projector  ?u  -  wA~1v?.  The  Moore  Penrose  inverse 
is  identical  to  the  true  inverse  for  nonsingular  aetrlees.  The  existence  of 
A'1  is  based  on  the  linear  independence  of  the  colunns  of  v.  The  Moore 
Penrose  inverse  X  satisfies  the  following  four  properties: 


AXA  - 

A 

(1) 

XAX  - 

X 

(2) 

<ax)t 

»  AX 

(3) 

T 

(XA) 

-  XA 

(A) 

An  exaaple  of  a  generalized  inverse  that  does  not  satisfy  the  four  conditions 

la 


-  (ZTA) 1  zT 


1 


This  inverse  satisfies  conditions  1,  2  and  4  but  not  3  as 


TIT  T  TIT 

0  -  A(ZAA)  Z  #  0  -  Z(A  Z)  A 

Condition  (3)  is  equivalent  to  requiring  A  A1  to  be  an  orthogonal 
projection.  The  oblique  projections  themselves  contain  a  generalized 
inverse.  The  inverse  A*  -  (Z^)1  ZT  is  called  a  1,  2,  4  inverse.1  In 
this  nomenclature  the  Moore  Penrose  inverse  is  a  1,  2,  3,  4  inverse. 

The  construction  of  Z  as 


T  -1  T 
A  -  B(B  B)  (B  A) 


insures  that  Zta  will  be  singular  only  if  the  subspaces  spanned  by  A  and  B 
are  not  independent.  If  the  subspaces  are  dependent  then  one  or  more  of  the 
columns  of  Z  will  be  of  zero  length.  Bor  nonsingular  ZTA,  the  generalized 
inverse  (Z^A)1  can  be  replaced  with  the  true  inverse  in  A*  and  0 
respectively.  Ib«  Moore  Penrose  Jnverse  is  identical  to  the  true  inverse  in 
chis  case, 
ve  have 


If  we  let  (Z^A)*  «  Y^Z^A)1  Y^  and  substitute  this  inverse  into  0 


TIT  TTITT 

0  -  A(ZAA)  za  -  a(yVa)  Y  Z 

using  the  Identity  (ZTA)(ZTA)"1  -  1  yields 

TTITT  T-1T  T-1T 

0  -  A(YiZiA)i(YiZ  A)(ZAA)  Z  -  AQ(Z  A)  1  ZA 

where  Q  ■  (Y^Z^A)^(Y^Z^A)  is  a  projection  operator. 

This  result  is  a  special  case  of  the  following.  A  generalized  inverse  of 
a  oonslngular  matrix  X  is  equal  to  the  product  of  an  idempotent  operator  and 
the  true  inverse 
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ESTIMATED  FRACTIONAL  CONTRIBUTIONS  TO  TOTAL  ABSORBANCE 
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for  £g  as  vas  obtained  in  Bq.  (7).  The  use  of  the  incorrect  weights  in  this 
model  has  not  had  an  adverse  effect  on  the  solution  variances.  The  weighting 
generates  a  numerically  stable  model  with  a  well  conditioned  normal  matrix 
which  is  easily  Inverted.  This  example  offers  some  hope  for  the  use  of 
weighting  schemes  which  minimire  apparent  correlation  or  minimize  Che 
importance  of  measurements  which  are  not  useful  for  classification.  Such 
schemes  are  of  obvious  use  in  spectral  pattern  recognition,  where  spectra  from 
different  classes  may  differ  significantly  only  in  a  few  bands.  This  can  be 
illustrated  with  a  comparison  of  aliphatic  hydrocarbons  and  aliphatic 
alcohols.  Figures  1  and  2  show  chat  several  features  are  essentlallv 
identical  and  hence  useless  .for  classification  purposes. 

The  uae  of  weights  in  oblique  projection  techniques  is  attractive.  The 
weighted  projection  method  can  be  formulated  so  that  once  the  projector  is 
generated  during  a  learning  step,  no  additional  burden  will  be  incurred.  The 
classification  step  is  as  simple  as  in  Che  unweighted  case.  The  form  of  the 
oblique  projection  operator  used  hers  is  0±  •  A(27a,)*z7  where  Z  is  a 
basis  for  the  projection  onto  A  which  is  orthogonal  to  That  is  ZTg  -  „ 

0  for  any  vector  B  that  belongs  to  +3.  When  using  weighted  measurements  a  Z 
is  sought  which  is  tf  orthogonal  to  $3.  In  the  unweighted  case  Z  is 
constructed  from 

T  -1  T 

Z  -  A  -  B(B  B)  BA 

m 

For  the  weighted  case  the  desired  Z  is 
Z  -  WA  -  WB(B^WB)  (A^VB) 

Then  Z^  -  ATWB  -  (B^WBKB^WB)  (A^VB)  -  0 


5 


contribute  to  an  increase  in  the  condition  number  and  to  the  correlation 
between  class  1  and  class  2. 


A  weighting  scheme  is  used  which  attempts  to  minimize  the  apparent 
correlation  between  classes  by  minimizing  the  condition  number.  The  following 
diagonal  weight  matrix  will  cause  the  apparent  correlation  to  be  zero. 


v  la  a  axm  diagonal  matrix.  The  model  now  trees  a  weighted  least  squares 
calculation.  The  matrix  (ATWA)  is  diagonal  and  class  1  appears  uncorrelated 
with  class  2.  The  condition  number  Cond(ATWA)  ■  1,  an  absolute  minimum.  We 
have 


and 


(ATWA)  - 


(ATWA) 


-1 


1/2 

0 


1/2 


!  1i  -  (A^WA)  A^Wb 


Since  these  weights  are  not  the  inverse  of  the  covariance  of  the  measurement 
errors,  the  covariance  of  X  must  be  calculated  from  Eq.  (2)  rather  than  Eq. 
(3)  and  in  principle  will  not  yield  the  minimum  variances.  However 
substitution  of  the  required  quantities  in  Eq.  (2)  yields  the  same  estimate 


average  vectors  for  class  1  and  class  2  respectively.  The  a  features  are 
measured  with  equal  precision  and  the  features  are  uncorrelated.  The 
covariance  matrix  of  the  errors  can  be  expressed  as  ~ I . 


•  • 

,  1  1 


Even  though  the  m  measurements  are  of  equal  precision,  only  the  first  will  be 
useful  in  distinguishing  between  the  two  classes.  For  the  minimum  distance 
classifier  Xx  >  Xj  if  b  belongs  to  class  1. 

For  this  example 


Che  condition  number  Cond(A^A)  -  a-1  and  the  covariance  can  be  calculated 
from  (4)  as 


,  _  -1  2  /  a 

The  uncertainties  in  Xj  and  Xj  are  obtained  from  the  diagonal  elements 


and  are  independent  of  a  for  large  a.  The  inclusion  of  a  large  number  of 
precisely  aeasured  but  unuseful  measurements  does  not  reduce  o^j.  but  does 


Gauss'  Theorem  stares  an  important  criteria  for  selection  of  weights. 
The  theorem  states  Chat  the  weight  which  will  give  minimum  variance  is  Che 
inverse  of  the  covariance  of  the  measurement  errors.  That  is  if  W  ■  Eg"1 
then  the  o2jj  will  be  a  minimum.  This  intuitively  makes  sense  as 
measurements  with  large  variances  will  have  small  weights  and  measurements 
with  small  variances  will  be  weighted  heavily.  The  minimum  variance 
covariance  matrix  for  Z  is  given  by 


-  cati;1at) 


-i 


If  Eg  •  cr*I  then 


I-  • 


c2(ATA) 


-1 


(3) 


(4) 


Eg  la  usually  not  known  and  ehe  assumption  of  It  being  a  constant 
matrix  a2!  is  ubiquitious  since  it  shows  no  prejudice  against  any 
measurements.  The  unweighted  least  squares  procedure  will  give  an  unbiased 
estimate  of  X  and  if  the  error  covariance  is  a  constant  matrix,  the  unweighted 
least  squares  will  give  a  minimum  variance  estimate  of  Z.  The  penalty  for 
using  the  wrong  weight  (V  a  Eg"1)  is  the  loss  of  minimum  variance.  A 
critical  question  concerning  minimum  variance  is  how  sensitive  is  the  variance 
to  the  wrong  weight.  (Mote:  no  weight  at  all  V  ■  1  is  the  wrong  weight  if 
Eg  is  not  a  constant  matrix.)  The  selection  of  measurements  or  weighting  of 
measurements  is  a  standard  approach  in  pattern  recognition.  The  weighting  or 
selection  is  based  more  on  usefulness  in  distinguishing  classes  rather  than  on 
concerns  about  measurement  error.  The  general  rather  than  accidental  success 
of  such  procedures  would  require  that  this  type  of  measurement  weighting  does 
not  have  a  large  effect  on  the  covariance.  A  simple  example  lends  support  to 
this  idea.  Co as id  ”  a  two  class  problem  and  a  simple  minimum  distance 
classifier  based  on  b  ■  Ax  where  A  is  m  z  2  whose  columns  and  a2  are  the 
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WEIGHTED  FEATURES  AND  SUBSPACE  METHODS 

The  incorporation  of  feature  weighting  into  oblique  projector*  was 
considered.  Justification  for  its  use  is  provided  below  from  a  standpoint  of 
least  squares  theory.  The  weighting  of  features  is  based  on  the  statistical 
description  of  the  measurement  errors.  The  assumptions  are  that  the  average 
values  of  errors  are  zero  and  that  the  variances  and  covariances  are  known. 

If  the  model  &  ■  b-Ax  is  adequate,  the  errors,  Rj,,  associated  with  each  row 
or  measurement  will  be  unbiased.  By  this  is  meant  that  with  an  ensemble  of 
repeated  measurement  of  b^  the  set  of  will  have  zero  mean.  Using  this 
ensemble  the  covariance  matrix  of  the  errors,  Ig,  can  be  formed.  For 
obvious  reasons  the  covariance  matrix  is  often  not  known.  In  any  event  the 
estimate  of  the  errors  in  the  solution  vector  x  is  related  to  the  covariance 
matrix,  Ig.  The  ensemble  of  solution  vectors  {Xj.}  has  a  covariance  matrix 


I  I 
A1  r^A1) 


where  A*  is  the  pseudoinverse  of  A.  For  unweighted  least  squares  the 
inverse  is  the  Moore  Penrose  inverse.  A*  ■  (A^A)"^*.  For  weighted 
least  squares  the  pseudoinverse  can  be  written  A*  -  (ATWA)“lATW,  where  W 
is  Che  weight  matrix.  The  weight  matrix  must  be  positive  definite.  The 
covariance  matrix  for  the  solution  vector,  Ig  is  in  Che  most  general 
formulation  given  by 


_  -1  _  _  -1 
Ix  -  (A  *A)  (at;  WA)  ( A  »A) 

It  is  the  diagonal  elements  of  £5,  cHjj,  that  give  the  errors  associated 
with  X 1 • 


1 


The  left  end  right  idempotent  operetora  ere  the  projectors  (XIX)  end  (XX1) 
respectively .  If  the  Moore  Penrose  Inverse  Is  used  the  Idempotent  operators 
ere  the  Identity  operators. 

The  projection  operators  developed  for  constrained  least  squares 
applications  are  also  expressible  In  terms  of  1,  2,  4  generalized  Inverses. 
The  problem  of  minimizing  AX  ■  b  subject  to  GX  »  0  had  the  solution  Pgb 
where 


T  -1  T 

PG  -  AQ(A  A)  A  and 

Q  -  1  -  (A^r1  GT[C(ATA)“l  CT]  G 

Substituting  H  -  (AV1)  Gt  and  St  •  G  yields  Q  •  1  -  H(STH)"1  ST 
with  the  1,  2,  4  generalized  inverse  (Sth“1)  ST. 

Reference 
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In  general  the  subspace  where  both  classes  overlap  strongly  is  all  that  is  needed 
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